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SIGNAL DETECTION BASED ON CHANNEL ESTIMATION 



BACKGROUND OF THE INVENTION 

Field of the Invention 

The present invention relates to signal processing, and, in particular, to the processing of noisy 
5 received signals, such as high-speed optical signals distorted by linear and non-linear polarization mode 
dispersion (PMD) effects during transmission through an optical transmission path and resulting in data 
pulse broadening and inter-symbol interference (ISI). 
Description of the Related Art 

As transmission speed increases in optical fiber communications, polarization mode dispersion 

10 (PMD) becomes a significant factor limiting system performance, especially for transmission speeds of 
10 Gb/s or higher. PMD causes data pulse broadening and creates inter-symbol interference (ISI). 
Unlike chromatic dispersion which can usually be countered by using a short dispersion-compensated 
fiber, PMD is time varying. While optical solutions have been proposed to counter PMD, they are 
usually very expensive and require a feedback path from the receiver back to the transmitter. 

15 Traditionally, electronic adaptive equalizers have been used to mitigate received signal distortion 

resulting in ISI. Well-known techniques for channels having linear distortion include linear feedforward 
equalization and linear decision feedback equalization (DFE). 

Fig. 1 shows a conventional equalizer 100 used to correct linear signal distortion. Equalizer 100 
has two adaptive equalizers: adaptive equalizer 102 configured to provide feedforward equalization and 

20 adaptive equalizer 104 configured to provide decision feedback equalization. In particular, adaptive 

equalizer 102 receives the current received signal and generates a linearly equalized signal v m \ This 
equalized signal v^' is presented to subtraction node 106, which subtracts the feedback signal generated 
by adaptive equalizer 104. The resulting difference signal is presented to slicer 108, which decides 
whether the current received signal v in corresponds to a "1" or a "0" by comparing the difference signal to 

25 a fixed threshold between 0 and 1 (e.g., half way between the reference voltages for logic "1" and logic 
"0" inside the slicer). The selected output data level is fed back to adaptive equalizer 104. In addition, 
the error signal (i.e., the difference between the difference signal generated at subtraction node 106 and 
the sliced signal level generated by slicer 108) is fed back to both adaptive equalizers 102 and 104 and 
used to dynamically control the coefficients within those equalizers using some conventional technique 

30 such as a least mean square (LMS) algorithm. Further information about typical implementations of 
adaptive equalizers 102 and 104 can be found in E.A. Lee and D.G. Messerschmitt, Digital 
Communication, Kluwer Academic Publisher, 1988, and S.U.H. Qureshi, "Adaptive Equalization," 
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Proceedings of the IEEE, Vol. 73, No. 9, September 1985, the teachings of both of which are 
incorporated herein by reference. 

When the channel is non-linear, such as an optical fiber channel dominated by PMD effects, non- 
linear equalization is used, since the effectiveness of equalizers, such as that shown in Fig. 1, is limited in 
5 those cases. Non-linear equalization techniques range from extremely complex solutions, such as those 
described in U.S. Patent No. 4,213,095, to simple single-tap non-linear DFE modules, such as those 
described in U.S. Patent No. 5,191,462. 

In general, effective non-linear equalization is a very complex and difficult process, involving the 
inversion of the non-linear channel response such that the combined channel and non-linear equalizer 
10 frequency response is flat. The optimization cost functions are often not smooth convex functions and, 
as a result, considerable adaptation convergence difficulties exist. These difficulties are manifested by 
the complexity of the techniques described in U.S. Patent No. 4,213,095. 

SUMMARY OF THE INVENTION 
The present invention is directed to a receiver that relies on channel estimation to mitigate inter- 

1 5 symbol interference (ISI) problems, especially when the channel is non-linear. When channel non- 
linearity is predominantly quadratic, a channel estimation-based receiver of the present invention 
performs extremely well with much lower complexity than traditional non-linear equalization techniques. 
The adaptive electronic solution of the present invention provides a versatile and cost-effective technique 
for correcting polarization mode dispersion (PMD) problems in optical transmission systems. One 

20 possible application of the present invention is to mitigate fiber PMD in high-speed SONET systems, 
such as OC 1 92, especially where second-order PMD, also known as polarization- induced chromatic 
dispersion (PCD), is strong. 

In one embodiment, the present invention is a receiver for a received signal having two or more 
data levels, the received signal having been transmitted over a transmission channel, the receiver 

25 comprising (a) two or more channel estimators, at least one channel estimator for each different data 

level for the received signal, each channel estimator being configured to model the transmission channel 
to generate an estimated signal corresponding to one of the data levels; and (b) a comparator configured 
to (1) receive the received signal and the estimated signal from each channel estimator and (2) select an 
output data level for the received signal. 

30 In another embodiment, the present invention is a method for processing a received signal having 

two or more data levels, the received signal having been transmitted over a transmission channel, the 
method comprising the steps of (a) generating at least one estimated signal for each data level based on a 
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model of the transmission channel; and (b) processing the received signal and the estimated signal for 
each data level to select an output data level for the received signal 

BRIEF DESCRIPTION OF THE DRAWINGS 
Other aspects, features, and advantages of the present invention will become more fully apparent 
5 from the following detailed description, the appended claims, and the accompanying drawings in which: 
Fig. 1 shows a conventional equalizer used to correct linear signal distortion; 
Fig. 2 shows a block diagram of a channel estimation-based receiver, according to one 
embodiment of the present invention, in which the received signal has two possible levels; and 

Fig. 3 shows a block diagram corresponding to a shared-component implementation of the 
1 0 channel estimation-based receiver of Fig. 2. 

DETAILED DESCRIPTION 
Fig. 2 shows a channel estimation-based receiver 200, according to one embodiment of the 
present invention. According to this embodiment, receiver 200 is designed to correct non-linear 
distortion for a received signal v m having two different data levels corresponding to logic "1" and logic 
15 "0". Receiver 200 has two quadratic (i.e., 2 nd order, non-linear) channel estimators: channel estimator 
201 for level "1" and channel estimator 202 for level "0", where each channel estimator adaptively 
generates an estimate for one of the two data levels for the current received signal v m . 

The two estimated signals are present to a comparator comprising two subtraction nodes 215 and 
216 and a compare-and-select module 217. In particular, each estimated signal is subtracted from the 
20 current received signal v in at one of the subtraction nodes and the two resulting difference signals are 
compared at compare-and-select module 217, which decides whether the current received signal 
corresponds to a level "1" or a level "0" (e.g., based on which difference signal has the smallest absolute 
value). 

Level "1" channel estimator 201 of receiver 200 comprises an adaptive equalizer 203 configured 
25 to receive a fixed reference signal v Y corresponding to an ideal current level "1" signal. In addition, 

adaptive equalizer 203 receives feedback signals from compare-and-select module 217 corresponding to 
both the previous data output signal(s) as well as the previous error signal(s), where the error signal is 
used to dynamically control the coefficients within adaptive equalizer 203, e.g., using a conventional 
LMS control technique. Adaptive equalizer 203 is preferably implemented based on conventional 
30 adaptive equalizer designs similar to that used for adaptive equalizer 104 of Fig. 1, since, in adaptive 
equalizer 203, the tap data are sliced symbols. 
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Adaptive equalizers are typically implemented as finite impulse response (FIR) taps in hardware, 
which consists of multipliers, accumulators, and storage devices to implement equations like 
(<3 k *coeff(l) + tf k _,*coeff(2) + ...), where a k and a k . x are current and previous inputs. For typical 
feedforward equalizers, where inputs represent either analog or digital input voltages, the required 
5 resolution is high. For feedback equalizers, inputs are sliced symbols (e.g., 0 or 1 in 2-level cases). 
Since only 1-bit resolution is needed, the multiplier design can be significantly simplified. 

The output signal x x from adaptive equalizer 203 is processed in parallel through two processing 
paths within channel estimator 201: a 1 st order path and a 2 nd order path. The 1 st order path multiplies the 
output signal x x by a 1 st order coefficient a x at multiplication node 207, while the 2 nd order path squares 
10 the output signal x x at squaring node 209 and multiplies the resulting squared signal x x by a 2 nd order 
coefficient a 2 at multiplication node 211. Channel estimator 201 also has a 0 th order path, which 
multiplies a fixed reference signal v ref by a 0 th order coefficient a 0 at multiplication node 205. The 
outputs from all three multiplication nodes are summed at summation node 213 to generate an estimated 
level "1" signal v, 6 * corresponding to Equation (1) as follows: 

15 Vy st = a 0 + a x x x + a 2 x\ (1) 

The estimated level "1" signal ( v\ st ) is then subtracted from the current received signal v in at subtraction 

node 215 to generate the difference signal for level "1" channel estimator 201 that is input to compare- 
and-select module 217. 

According to the present invention, each of multiplication nodes 205, 207, and 211 are 

20 dynamically controlled by the error signal generated by compare-and-select module 217 such that the 

coefficients a 0 , a 1? and oc 2 are all adaptive coefficients that are dynamically adjusted, e.g., using an LMS 
control technique. (Note that, in Fig. 2, the thick arrow is used to signify that the error signal is input to 
each of multiplication nodes 205, 207, and 211.) In preferred implementations, the 1 st order coefficient 
a, has a fixed value of L As such, the 1 st order path can be implemented simply as a signal path 

25 connecting the output of adaptive equalizer 203 directly to the corresponding input of summation node 
213 without requiring multiplication node 207. 

Level "0" channel estimator 202 is analogous to level "1" channel estimator 201, with elements 
204, 206, 208, 210, 212, and 214 of level "0" channel estimator 202 configured to operate in analogous 
fashion to elements 203, 205, 207, 209, 211, and 213 of level "1" channel estimator 201, to generate the 

30 estimated level "0" signal, which is subtracted from the current received signal v in at subtraction node 
216 to generate the difference signal for level "0" channel estimator 202 that is input to compare-and- 
select module 217. Note that the input to level "0" channel estimator 202 is a fixed, level "0" reference 



IDS# 122582 (990.0320) 



-4- 



Kim6-8 



signal v 0 , which is different from the fixed, level "1" reference signal v,. The resulting estimated level 
"0" signal generated by channel estimator 202 can be represented by 

v e o St =Po+Axo+&4 (2) 

where jc 0 is the output signal from adaptive equalizer 204 and p o , P 1? and P 2 are the adaptive 0 th , 1 st , and 
5 2 nd order coefficients applied at multiplication nodes 206, 208, and 212, respectively, and dynamically 
controlled based on the error signal generated by compare-and-select module 217. 

By including the 2 nd order terms ( a 2 x^ and J3 2 Xq ), channel estimators 201 and 202 model a 

general 2 nd order system. If the transmission channel for received signal is dominated by 1 st and 2 nd 
order effects, then channel estimators 201 and 202 should produce accurate replicas of the two possible 

1 0 channel outputs after they have been properly adapted to model the channel. The present invention 

avoids the typically difficult problem associated with conventional non-linear equalization schemes of 
having to invert the channel response. 

Fig. 3 shows a block diagram corresponding to a preferred shared-component implementation of 
the channel estimation-based receiver 200 of Fig. 2. According to this shared-component 

15 implementation, receiver 300 is implemented with only a single adaptive equalizer 302, which differs 
from adaptive equalizers 203 and 204 (e.g., the first tap corresponding to the current received signal is 
stripped). The output signal from adaptive equalizer 302 is used to generate the input signals for both 
channel estimators 201 and 202. 

In particular, the level "1" reference signal v l is multiplied by a coefficient c x (e.g., corresponding 

20 to the first tap of a conventional adaptive equalizer) at multiplication node 303 and the resulting product 
(v^c,) is added to the output signal from adaptive equalizer 302 at summation node 305 to generate the 
input signal to channel estimator 201. Analogously, the level "0" reference signal v 0 is multiplied by the 
coefficient c { at multiplication node 304 and the resulting product (Vo*^) is added to the output signal 
from adaptive equalizer 302 at summation node 306 to generate the input signal to channel estimator 202. 

25 Although the two channel estimators (e.g., 201 and 202 of Figs. 2 and 3) are shown with different 

sets of coefficients (a 0 , a b a 2 ) and (P 0 , P 2 ), in some implementations, the different channel estimators 
could be implemented using a single set of coefficients. In that case, the potential exists for even further 
sharing of components, where two or more channel estimators temporally share the same set of 0 th , 1 st , 
and 2 nd order path components, with different inputs being applied at different times corresponding to 

30 different channel estimators. Such temporal sharing would add latency to and decrease the speed (i.e., 
data throughput) of the receiver, which could make such an implementation unacceptable for certain 
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applications. Another way of sharing is to share the LMS update circuit for the coefficients and just copy 
the updated results for the two channel estimators. 

Channel estimation-based receivers 200 and 300 may be implemented in software or in hardware, 
in either the digital domain or the analog domain, or in combinations thereof. For high-speed 
5 applications (i.e., data rates as high as 1 0 Gb/s or even higher), the receivers are preferably implemented 
in hardware in the analog domain. 

Channel estimation-based receivers 200 and 300 are designed with the following characteristics: 
o The received signal v m has only two data levels ("1" and "0"); 
o The channel estimators contain only 0 th , 1 st , and 2 nd order terms; and 
1 0 o Only the current and past data signals are fed into the adaptive equalizers). 

Those skilled in the art will appreciate that the present invention can be implemented for applications 
having other characteristics. 

For example, channel estimation-based receivers of the present invention can be implemented for 
applications in which the received signal has more than two data levels. In that case, the receiver is 
1 5 implemented with more than two channel estimators: (at least) one channel estimator for each different 
data level, where the estimated signal generated by each channel estimator is subtracted from the current 
received signal to generate a difference signal for the compare-and-select module, which still preferably 
selects the output data level for the current received signal based on the difference signal having the 
smallest absolute value. 

20 Furthermore, channel estimation-based receivers of the present invention can be implemented for 

applications in which the channel estimators contain one or more additional, higher-order terms beyond 
the 2 nd order term. In that case, each channel estimator will have an additional processing path for each 
additional, higher-order term comprising, for example, a sequence of multiplication nodes to generate the 
higher-order term, where one of the multiplication nodes is an adaptive node corresponding to a 

25 dynamically controlled coefficient for the higher-order term. 

Moreover, channel estimation-based receivers of the present invention can be implemented for 
applications in which each adaptive equalizer receives future data as well as the past and current data as 
tap inputs. In that case, the receiver should be implemented with a different channel estimator for each 
different combination of current and future data. For example, for a receiver designed for a received 

30 signal having two data levels "1" and "0" where the adaptive equalizers take into account the current data 
value and one future data value, the receiver will preferably have four different channel estimators: one 
for each different combination of possible current and future data values (i.e., 00, 01, 10, and 1 1). If the 
adaptive equalizers are configured to take into account two future data values, then the receiver will 



IDS#122582(990 0320) 



-6- 



Kim6-8 



preferably have eight different channel estimators corresponding to the eight different combinations of 
one current and two future data values. 

These different characteristics can be combined such that, in general, a channel estimation-based 
receiver according to the present invention will preferably have m"* 1 channel estimators: one channel 
5 estimator for each different possible combination of the current data value and n future data values, 

where each data value has m different possible levels, where m is an integer greater than 1 and n is a non- 
negative integer, and each channel estimator has p+\ processing paths corresponding to (0*, 1 st , 2 nd , 
/7 th ) order terms, where p is an integer greater than 1. In preferred shared-component implementations, 
different channel estimators will share adaptive equalizers. In one possible shared-component 

10 implementation, all nf* 1 channel estimators will share a single adaptive equalizer with suitable 

components provided to generate the appropriate input signal for each different channel estimator from 
the single output signal generated by the shared adaptive equalizer. 

Although the present invention has been described in the context of processing electrical signals 
corresponding to optical signals transmitted over optical fiber transmission channels, the present 

1 5 invention can also be applied to signals received from other types of transmission channels, including 
electrical or wireless (i.e., "over-the-air") transmission channels. 

The present invention may be implemented as circuit-based processes, including possible 
implementation on a single integrated circuit. As would be apparent to one skilled in the art, various 
functions of circuit elements may also be implemented as processing steps in a software program. Such 

20 software may be employed in, for example, a digital signal processor, micro-controller, or general- 
purpose computer. 

The present invention can be embodied in the form of methods and apparatuses for practicing 
those methods. The present invention can also be embodied in the form of program code embodied in 
tangible media, such as floppy diskettes, CD-ROMs, hard drives, or any other machine-readable storage 

25 medium, wherein, when the program code is loaded into and executed by a machine, such as a computer, 
the machine becomes an apparatus for practicing the invention. The present invention can also be 
embodied in the form of program code, for example, whether stored in a storage medium, loaded into 
and/or executed by a machine, or transmitted over some transmission medium or carrier, such as over 
electrical wiring or cabling, through fiber optics, or via electromagnetic radiation, wherein, when the 

30 program code is loaded into and executed by a machine, such as a computer, the machine becomes an 

apparatus for practicing the invention. When implemented on a general-purpose processor, the program 
code segments combine with the processor to provide a unique device that operates analogously to 
specific logic circuits. 



IDS# 122582 (990.0320) 



-7- 



Kim 6-8 



It will be further understood that various changes in the details, materials, and arrangements of 
the parts which have been described and illustrated in order to explain the nature of this invention may be 
made by those skilled in the art without departing from the scope of the invention as expressed in the 
following claims. 
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CLAIMS 

What is claimed is: 

1 LA receiver for a received signal having two or more data levels, the received signal having been 

2 transmitted over a transmission channel, the receiver comprising: 

3 (a) two or more channel estimators, at least one channel estimator for each different data level for 

4 the received signal, each channel estimator being configured to model the transmission channel to 

5 generate an estimated signal corresponding to one of the data levels; and 

6 (b) a comparator configured to ( 1 ) receive the received signal and the estimated signal from each 

7 channel estimator and (2) select an output data level for the received signal. 

1 2. The receiver of claim 1 , wherein each channel estimator implements a 2 nd order or higher model 

2 of the transmission channel. 

1 3 . The receiver of claim 2, wherein the model is an adaptive model of the transmission channel that 

2 is dynamically controlled based on an error signal generated by the comparator. 

1 4. The receiver of claim 2, wherein each channel estimator comprises a processing path for each 

2 order term in the model of the transmission channel. 

1 5. The receiver of claim 4, wherein at least one of the processing paths in each channel estimator 

2 comprises a multiplication node having an adaptive coefficient that is dynamically controlled based on an 

3 error signal generated by the comparator. 

1 6. The receiver of claim 5, wherein a processing path in each channel estimator corresponding to a 

2 1 st order term of the model with a coefficient having a value of 1, wherein the 1 st order term processing 

3 path is implemented without a multiplication node. 

1 7. The receiver of claim 1, wherein the two or more channel estimators comprise one or more 

2 adaptive equalizers, each adaptive equalizer configured to receive a current data level corresponding to 

3 one of the data levels and to generate an input signal for one or more of the channel estimators. 

1 8. The receiver of claim 7, wherein at least one adaptive equalizer is shared by two or more of the 

2 channel estimators. 
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1 9. The receiver of claim 8, wherein all of the channel estimators share a single adaptive equalizer. 

1 10. The receiver of claim 7, wherein each adaptive equalizer is further configured to receive one or 

2 more future data levels and the receiver comprises a channel estimator for each different combination of 

3 current and future data levels. 

1 11. The receiver of claim 7, wherein tap data for each adaptive equalizer corresponds to sliced 

2 symbols corresponding to the two or more data levels. 

1 12. The receiver of claim 1, wherein the comparator comprises: 

2 (a) a subtraction node for each channel estimator configured to generate a difference signal between 

3 the received signal and the corresponding estimated signal; and 

4 (b) a compare-and-select module configured to receive the difference signals from the subtraction 

5 nodes and to select the output data level for the received signal based on a difference signal having a 

6 smallest absolute value. 

1 13. The receiver of claim 1, wherein: 

2 the transmission channel is an optical transmission channel; and 

3 the two or more channel estimators and the comparator are implemented in a single integrated circuit 

4 as analog circuitry. 

1 14. A method for processing a received signal having two or more data levels, the received signal 

2 having been transmitted over a transmission channel, the method comprising the steps of: 

3 (a) generating at least one estimated signal for each data level based on a model of the transmission 

4 channel; and 

5 (b) processing the received signal and the estimated signal for each data level to select an output data 

6 level for the received signal. 

1 15. The method of claim 14, wherein step (a) comprises the step of implementing a 2 nd order or 

2 higher model of the transmission channel. 

1 16. The method of claim 15, wherein the model is an adaptive model of the transmission channel that 

2 is dynamically controlled based on an error signal generated during step (b). 
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1 17. The method of claim 14, further comprising the steps of: 

2 (c) generating a difference signal between the received signal and the corresponding estimated 

3 signal; and 

4 (d) selecting the output data level for the received signal based on a difference signal having a 

5 smallest absolute value. 
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SIGNAL DETECTION BASED ON CHANNEL ESTIMATION 



ABSTRACT OF THE DISCLOSURE 
A receiver for a received signal having two or more different data levels comprises two or more 
channel estimators, (at least) one channel estimator for each different data level, where each channel 
estimator preferably implements an adaptive 2 nd order or higher model of the transmission channel over 
which the received signals was transmitted to generate an estimated signal for one of the different data 
levels. The receiver also has a comparator that compares the current received signal to the estimated 
signals generated by the different channel estimators to select an output data value for the current 
received signal. The adaptive model of the transmission channel has coefficients that are dynamically 
controlled based on an error signal generated by the comparator. Each channel estimator relies on an 
output signal generated by an adaptive equalizer. In preferred shared-component implementations, each 
adaptive equalizer is shared by two or more different channel estimators, and, in one possible preferred 
shared-component implementation, all of the different channel estimators share a single adaptive 
equalizer. 
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(Reg. No. 38173) 
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(Reg. No. 38944) 
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Claude R. Narcisse (Reg. No. 3 8979) 

Joseph J. Opalach (Reg. No. 36229) 

Neil R. Ormos (Reg. No. 35309) 

Eugen E. Pacher (Reg. No. 29964) 

Jack R. Penrod (Reg. No. 3 1 864) 

Gregory C. Ranieri (Reg. No. 29695) 

Scott J. Rittman (Reg. No. 39010) 

Ferdinand M. Romano (Reg. No. 32752) 

Eugene J. Rosenthal (Reg. No. 36658) 

Bruce S. Schneider (Reg. No. 27949) 

Ronald D. Slusky (Reg. No. 26585) 

David L. Smith (Reg. No. 30592) 

Ozer M. N. Teitelbaum (Reg. No. 36698) 

John P. Veschi (Reg. No. 39058) 

David Volejnicek (Reg. No. 29355) 

Charles L. Warren (Reg. No. 27407) 

Jeffrey M. Weinick (Reg. No. 36304) 

Eli Weiss (Reg. No. 17765) 



I hereby appoint the attorney(s) on ATTACHMENT A as associate attorney(s) in the 
aforementioned application, with full power solely to prosecute said application, to make alterations 
and amendments therein, to receive the patent, and to transact all business in the Patent and 
Trademark Office connected with the prosecution of said application. No other powers are granted 
to such associate attorney(s) and such associate attorney(s) are specifically denied any power of 
substitution or revocation. 

Full name of 1 st inventor: Helen H. Kim 

Inventor's signatu re Mj2-^/4^l<^ Dat e ^ l ^-^q ) 
Residence: Holmdel. Monmouth County. New Jersey 
Citizenship: USA 

Post Office Address: 1 1 Galewood Drive 

Holmdel. New Jersey 07733 
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Full name of 2 nd inventor: Meng-Lin Yu 

Inventor's signature ^ ^ Date ^/i^j/z*. 

Residence: Morganville. Monmouth CountWfsfew Jersey 
Citizenship: USA 

Post Office Address: 135 Yorkshire Drive 

Morganville. New Jersey 07751 
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ATTACHMENT A 

Attorney Name(s): Steve Mendelsohn Reg. No.: 35.951 

Tan M. Hushes 41.083 

Telephone calls should be made to Mendelsohn & Associates, P.C. at: 
Phone No.: 215-557-6657 
Fax No.: 215-557-8477 



All written communications are to be addressed to: 

Steve Mendelsohn 
Mendelsohn & Associates, P.C. 
1515 Market Street, Suite 715 
Philadelphia, PA 19102 

Customer No.: 22186 



990.0320 



